Reject serde-reserved keys in HITL params_input by haseebmalik18 · Pull Request #71054 · apache/airflow

haseebmalik18 · 2026-08-04T05:00:24Z

A HITL response whose params_input contains a serde-reserved key (__classname__ or __id__) cannot be serialized when the task resumes. Today the response is silently discarded and the user has no way to resubmit a corrected one.

This validates params_input when the response is submitted and returns a 400 if it carries one of those keys, before anything is written. The task stays in AWAITING_INPUT, so the user can fix the input and resubmit.

Only the public HITL route needs this. The execution API route builds its params_input from trusted operator code rather than user input, so it is left as is.

closes: #71036

amoghrajesh · 2026-08-04T08:06:51Z

This doesn't have to be in 3.3.1, but if we are comfortable (cc @Lee-W) we can consider it.

Lee-W · 2026-08-04T08:24:13Z

A few nitpicks, but overall this looks good. I also don't think it's worth blocking the 3.3.1 release.

vatsrahul1001 · 2026-08-04T08:34:20Z

Moved to 3.3.2

…ction

haseebmalik18 · 2026-08-05T18:21:39Z

@Lee-W Applied changes left from review

jason810496

Thanks for fixing this.

jason810496 · 2026-08-06T06:41:27Z

 log = structlog.get_logger(__name__)


+# Keys that ``serde.serialize`` refuses at any depth. A ``params_input`` carrying one could not be


IMO, we should follow the convetion in XCom and validate at the data model layer:

airflow/airflow-core/src/airflow/api_fastapi/core_api/datamodels/xcom.py

Lines 87 to 119 in 9a85faa

def _check_forbidden_xcom_keys(value: Any) -> Any:

"""Recursively reject forbidden deserialization keys in user-provided XCom data."""

from airflow._shared.serialization import FORBIDDEN_XCOM_KEYS

def _walk(obj: Any, path: str = "value") -> None:

if isinstance(obj, str):

# A value submitted as a JSON string literal (e.g. ``json.dumps({...})``)

# is stored verbatim and re-parsed into a dict/list on a

# ``deserialize=true`` read, which would otherwise smuggle reserved keys

# past the dict/list checks below. Re-parse and inspect the decoded

# structure the same way the read path does.

try:

decoded = json.loads(obj)

except (ValueError, TypeError):

return

if isinstance(decoded, (dict, list)):

_walk(decoded, path)

return

if isinstance(obj, dict):

found = FORBIDDEN_XCOM_KEYS & obj.keys()

if found:

raise ValueError(

f"XCom {path} contains reserved serialization keys: {', '.join(sorted(found))}. "

f"These keys are reserved for internal use."

)

for k, v in obj.items():

_walk(v, f"{path}.{k}")

elif isinstance(obj, (list, tuple)):

for i, item in enumerate(obj):

_walk(item, f"{path}[{i}]")

_walk(value)

return value

airflow/shared/serialization/src/airflow_shared/serialization/__init__.py

Lines 20 to 44 in 9a85faa

CLASSNAME = "__classname__"

VERSION = "__version__"

DATA = "__data__"

SCHEMA_ID = "__id__"

CACHE = "__cache__"

# Legacy serialization format keys (processed by serde for backwards compatibility)

OLD_TYPE = "__type"

OLD_SOURCE = "__source"

OLD_DATA = "__var"

OLD_DICT = "dict"

FORBIDDEN_XCOM_KEYS = frozenset(

{

CLASSNAME,

VERSION,

DATA,

SCHEMA_ID,

CACHE,

OLD_TYPE,

OLD_SOURCE,

OLD_DATA,

}

)

Reject serde-reserved keys in HITL params_input apache#71036

709d038

haseebmalik18 requested review from bugraoz93, choo121600, ephraimbuddy, henry3260, jason810496, pierrejeambrun, rawwar and shubhamraj-git as code owners August 4, 2026 05:00

boring-cyborg Bot added the area:API Airflow's REST/HTTP API label Aug 4, 2026

Merge branch 'main' into hitl-reject-serde-reserved-param-keys

9e9c0de

haseebmalik18 changed the title ~~Reject serde-reserved keys in HITL params_input #71036~~ Reject serde-reserved keys in HITL params_input Aug 4, 2026

eladkal added this to the Airflow 3.3.1 milestone Aug 4, 2026

eladkal added type:bug-fix Changelog: Bug Fixes backport-to-v3-3-test Backport to v3-3-test labels Aug 4, 2026

eladkal requested a review from potiuk August 4, 2026 07:22

vatsrahul1001 requested a review from Lee-W August 4, 2026 07:52

Lee-W reviewed Aug 4, 2026

View reviewed changes

Comment thread airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_hitl.py Outdated

vatsrahul1001 modified the milestones: Airflow 3.3.1, Airflow 3.3.2 Aug 4, 2026

haseebmalik18 and others added 2 commits August 4, 2026 14:22

Share HITL reserved-key constant and document the params_input restri…

e2e3c04

…ction

Merge branch 'main' into hitl-reject-serde-reserved-param-keys

0625933

haseebmalik18 closed this Aug 4, 2026

haseebmalik18 reopened this Aug 4, 2026

haseebmalik18 added 2 commits August 4, 2026 21:05

Merge branch 'main' into hitl-reject-serde-reserved-param-keys

362bf41

Merge branch 'main' into hitl-reject-serde-reserved-param-keys

5a3ccbf

jason810496 reviewed Aug 6, 2026

View reviewed changes

Navigation Menu

Uh oh!

Reject serde-reserved keys in HITL params_input - #71054

Reject serde-reserved keys in HITL params_input#71054
haseebmalik18 wants to merge 6 commits into
apache:mainfrom
haseebmalik18:hitl-reject-serde-reserved-param-keys

haseebmalik18 commented Aug 4, 2026 •

edited

Loading

Uh oh!

amoghrajesh commented Aug 4, 2026

Uh oh!

Uh oh!

Lee-W commented Aug 4, 2026

Uh oh!

vatsrahul1001 commented Aug 4, 2026

Uh oh!

haseebmalik18 commented Aug 5, 2026 •

edited

Loading

Uh oh!

jason810496 left a comment

Uh oh!

jason810496 Aug 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

		log = structlog.get_logger(__name__)


		# Keys that ``serde.serialize`` refuses at any depth. A ``params_input`` carrying one could not be

	def _check_forbidden_xcom_keys(value: Any) -> Any:
	"""Recursively reject forbidden deserialization keys in user-provided XCom data."""
	from airflow._shared.serialization import FORBIDDEN_XCOM_KEYS

	def _walk(obj: Any, path: str = "value") -> None:
	if isinstance(obj, str):
	# A value submitted as a JSON string literal (e.g. ``json.dumps({...})``)
	# is stored verbatim and re-parsed into a dict/list on a
	# ``deserialize=true`` read, which would otherwise smuggle reserved keys
	# past the dict/list checks below. Re-parse and inspect the decoded
	# structure the same way the read path does.
	try:
	decoded = json.loads(obj)
	except (ValueError, TypeError):
	return
	if isinstance(decoded, (dict, list)):
	_walk(decoded, path)
	return
	if isinstance(obj, dict):
	found = FORBIDDEN_XCOM_KEYS & obj.keys()
	if found:
	raise ValueError(
	f"XCom {path} contains reserved serialization keys: {', '.join(sorted(found))}. "
	f"These keys are reserved for internal use."
	)
	for k, v in obj.items():
	_walk(v, f"{path}.{k}")
	elif isinstance(obj, (list, tuple)):
	for i, item in enumerate(obj):
	_walk(item, f"{path}[{i}]")

	_walk(value)
	return value

	CLASSNAME = "__classname__"
	VERSION = "__version__"
	DATA = "__data__"
	SCHEMA_ID = "__id__"
	CACHE = "__cache__"

	# Legacy serialization format keys (processed by serde for backwards compatibility)
	OLD_TYPE = "__type"
	OLD_SOURCE = "__source"
	OLD_DATA = "__var"
	OLD_DICT = "dict"

	FORBIDDEN_XCOM_KEYS = frozenset(
	{
	CLASSNAME,
	VERSION,
	DATA,
	SCHEMA_ID,
	CACHE,
	OLD_TYPE,
	OLD_SOURCE,
	OLD_DATA,
	}
	)

Uh oh!

Conversation

haseebmalik18 commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

amoghrajesh commented Aug 4, 2026

Uh oh!

Uh oh!

Lee-W commented Aug 4, 2026

Uh oh!

vatsrahul1001 commented Aug 4, 2026

Uh oh!

haseebmalik18 commented Aug 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jason810496 left a comment

Choose a reason for hiding this comment

Uh oh!

jason810496 Aug 6, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

haseebmalik18 commented Aug 4, 2026 •

edited

Loading

haseebmalik18 commented Aug 5, 2026 •

edited

Loading